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Rolsert  Kalaba* 
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Santa  MDnlca,  California 


SUM4ARY 

Several  Important  classes  of  problems  in  the  theory  of  automatic 
control- -Including  time-optimal  control — find  a  natural  setting  in 
the  field  of  grajih  theory.  Various  aspects  of  formulation,  ana- 
lytlceil  and  numerical  treatment,  and  isplementatlon  are  sketched, 
nie  paper  is  intended  to  be  quite  self-contained. 

1.  Introduction 

A  problem  of  centsral  isiportance  in  the  theory  of  automatic  con¬ 
trol  involves  the  transforming  of  a  system  from  an  initial  state 
into  a  desired  terminal  state  in  the  most  efficient  fashion.  If, 
for  example  a  gust  of  vlnd  causes  an  aircraft  to  begin  rolling,  the 
automatic  pilot  is  supposed  to  send  c<»itrol  signals  to  the  control 
surfaces  in  an  effort  to  restore  the  craft,  as  rapidly  as  possible, 
to  the  horizontal  position  with  no  angular  velocity  about  its  longi¬ 
tudinal  axis.  Or  consider  a  spacecraft  which  is  about  to  re-enter 
the  earth's  atmosphere.  It  is  desired  to  fly  along  a  trajectory 
which  will  bring  the  craft  to  the  surface  of  the  earth  while  keeping 
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the  maximum  ten5)erature  to  which  the  surface  of  the  craft  is  exx>osed 
as  low  as  possible.  One  might  also  wish  to  select  a  path  which  will 
minimize  the  maximum  deceleration  during  descent. 

From  the  mathematiced  viewpoint  it  seems  natural  to  consider 
such  problems  as  belonging  to  the  calculus  of  veu:lations.  Toward 
this  end  one  introduces  a  state  vector  x(t),  with  x(0)  =  c,  a 
control  vector  y(t),  euid  a  dynamical  equation 

(1)  X  =  f(x,y). 

We  wish  to  determine  the  control  vector  y(t)  in  such  a  manner  that 
the  system  is  transformed  from  the  initial  state  c  to  some  desired 
termlnad  state,  say  x  =  0,  as  rapidly  as  possible.  An  extensive 
literature  now  exists  concerning  such  problems  [1,2],  especially  for 
the  case  in  which  the  function  f  is  linear  in  x  and  y  and  the 
ccmponents  of  the  control  vector  y  are  subject  to  certain  con¬ 
straints.  Alternatively,  we  might  wish  to  determine  the  control 
vector  y(t)  so  that  we  minimize  the  maximum  value  of  some  function 
g(x(t),y(t))  during  the  course  of  transforming  the  system  frcm  the 
initial  state  to  the  terminal  state.  There  is  a  less  extensive 
literature  associated  with  such  problems  [3^^]» 

If  we  keep  in  mind  that  the  treatment  of  significant  problems 
in  this  area  will  ultimately  involve  the  use  of  digital  computers, 
wherein  all  variables  are  rendered  discrete,  and  that  some  control 
systems  operate  in  a  discrete  fashion  by  design,  it  becomes  of  in¬ 
terest  to  consider  a  discrete  formulation  and  treatment  of  automatic 
control  processes,  as  opposed  to  a  continuous  one.  The  aim  of  this 
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paper  is  to  give  an  Indication  of  the  current  state  of  affairs  In 
this  program. 

2.  Time-optimal  Control  aind  the  Nature  of  Feedback 

Let  us  consider  a  system  S  which  may  he  found  in  any  of  a 
finite  number,  N,  of  possible  states.  We  represent  these  states 
by  the  nodes  of  a  graph  [5,6]  and  nvanber  them  1,2,  ...,N.  In  the 
course  of  time  the  system  changes  its  state;  i.e.,  it  undergoes  a 
sequence  of  transformations  which  we  call  a  process.  When  the  system 
is  in  state  1,  we  assume  that  a  control  decision  can  be  made,  the 
resiilt  of  which  is  that  the  system  is  transformed  into  some  new  state. 
In  general,  when  the  system  is  in  state  i,  only  certain  new  states 
may  be  attained  as  the  resiilt  of  making  a  control  decision. 

In  the  passage  frcxa  state  1  to  state  J  a  certain  amount  of 
a  resource  such  as  energy  must  be  consumed  during  such  a  transforma¬ 
tion.  This  assigns  a  number,  t^j,  to  the  directed  arc  from  i  to 
J.  Let  us  consider  that  we  wish  to  control  the  system  S  in  such 
a  way  that  if  the  system  is  disturbed  from  its  equllibrlm  (or  most 
desirable)  state,  say  the  state  N,  then  we  shall  return  it  to  the 
equilibrium  state  in  the  least  possible  time. 

At  first  glance  it  might  seem  that  we  wish  merely  to  find  a 
shortest  trajectory  through  phase  space  leading  from  the  initial 
state  i  to  the  desired  terminal  state  N.  Actually  this  is  not 
so.  Frequently  we  shall  have  no  prior  knowledge  of  what  the  particu¬ 
lar  initial  state  i  will  be,  so  that  we  must  be  prepared  to  trans¬ 
form  the  system  from  any  initial  state  i  into  the  desired  terminal 
state,  and  do  so  in  minimum  time.  Now  we  may  go  one  step  further 


8uid  note  that  if  the  system  is  in  state  i,  it  is  not  necessary  to 
specify  the  entire  trajectory  from  i  to  N.  Indeed,  all  ve  need 
do  is  specify  that  if  the  system  is  in  state  i,  then  the  next  state 
into  which  to  treinsform  the  system  is  a  particular  state  J. 

These  considerations  enable  us  to  distinguish  between  what  is 
celled  open -loop  and  feedback  control.  In  an  open -loop  process  the 
entire  sequence  of  transformations  is  specified  ediead  of  time.  This 
type  of  control  is  useful  when  one  has  great  confidence  that  the 
system  will  perform  as  specified  and  that  random  extemeLL  Influences 
cure  negligible.  In  practice  this  means  that  all  control  Influences 
will  be  exerted  in  a  prescribed  fashion  as  a  function  of  time,  i.e., 
according  to  the  reading  of  a  clock,  no  notice  being  taken  of  the 
actual  state  of  the  system.  A  simple  example  of  this  is  the  control 
of  the  temperature  of  a  room  by  turning  a  furnace  on  and  off  at 
certain  fixed  times.  Open-loop  control  Is  usually  economical  to 
provide,  but  In  turn,  asy  not  provide  satisfactory  control  due  to 
tbe  neglect  of  external  factors. 

A  more  sophisticated  type  of  control  Is  achieved  by  carrying 
out  the  following  cycle  of  operation:  (l)  determine  the  state  of 
the  system  (measure  the  tengwrature  of  the  room);  (2)  decide  on  a 
control  action  (turn  the  furnace  on  or  off);  (3)  return  to  (l). 

A  control  system  which  operates  on  this  principle  Is  termed  a  feed¬ 
back  control  system.  There  is  an  extensive  theory  of  such  processes 
going  back  to  Maxwell  [7>0^9]*  For  the  most  part  this  is  a  theory 
describing  how  a  system  would  act  If  controlled  in  a  certain  way, 
the  notion  of  stability  playing  a  major  role  [10] .  It  Is  clear, 
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though,  that  we  wish  to  control  in  optimal  fashion,  so  that  emphasis 
has  turned  from  stability  to  optimality  considerations,  though,  of 
course,  there  is  an  intimate  relation  between  the  two. 

3 .  Formulation  [11] 

We  introduce  the  following  nomenclature: 

(l)  A  “  time  required  to  transform  the  system  from  state 

^  J 

i  to  state  J  over  a  direct  link. 


(2)  '*i  “  transform  the  system  from  state  i  to 

state  N  (the  desired  state)  in  optimal  fashion, 
i  =  1,2, .. . ,N. 


To  derive  relations  among  these  quantities  we  note  that  if  the 
system  is  in  state  i  and  the  decision  to  transform  it  into  state 
J  is  made,  i  ^  i,  then  the  process  will  have  to  continue  optimally 
from  state  j  to  state  N  if  the  entire  process  is  to  be  optimal. 
Furthermore,  the  choice  of  the  next  state,  state  J,  is  made  on  the 
basis  of  minimizing  the  sum 


(3) 


‘ij  *  "y 


for  t  .  is  the  time  to  pass  directly  from  state  i  to  state  j, 

1 J 

«md  u.  is  the  minimal  time  required  to  pass  from  state  j  to  the 
J 

terminal  state  N.  These  observations,  manifestations  of  Bellman's 
principle  of  optimality  [l],  lead  to  the  nonlinear  system  of  equations 


(4)  u^  =  min  (t^j  ^  *  1,2,..., N  -  1, 

J 

“m  =  0- 
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4 .  Uniqueness  [U] 

Since  we  contemplate  solving  the  equations  (3.4)  via  various 
successive  approximation  techniques,  it  is  important  to  establish 
the  laniqueness  of  the  solution.  Suppose  that  (u^,)  and  re¬ 

present  two  solutions  of  the  system  of  equations  (3*4),  and  that  k 
is  an  index  where  the  difference 


(1) 


Ui  - 


achieves  its  maximum.  Let 


(2)  *  “jl  ■  *lcr  *  S' 
and 

(3)  S  ■  ^  fSj  *  “jJ  -  s.  ♦ 
Since  we  are  assuming  that 


(i*) 


Sj  =•  »> 


it  is  clesu:  that 


(5) 


r  ^  k,  e  fi  k. 


We  have  the  inequalities 


(6) 


\  “  ^kr  ^  '^r  ^  \s  ^  % 


(7) 


«k  =  \s  "  ^s  ^  \r  ^  "r^ 


which  lead  to  the  relation 
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(6)  -  U^. 


Since  k  is  an  index  for  which  the  difference  u 
mized,  equality  must  hold  in  relation  (8), 


U. 


is  maxi- 


(9) 

which  inqjlies  that  equality  also  holds  in  relation  (6) 


But  now  we  may  repeat  the  same  reasoning  for  the  state  s  and 
establish  that  there  is  another  state,  state  m,  for  which 

(11)  u  -  U  =  u  -  U  =  u.  -  U.  . 

'  m  m  s  s  K  k 

Furthermore  m  s  and  m  k  since 

(12)  u.  =  t.  „  +  t  +  u  . 

'  '  K  ks  sm  m 

Continuing  in  this  way  we  must  eventually  ccae  upon  the  state  N, 
for  which 


(13)  Ujj  -  Uj^  =  0, 

which  completes  the  proof. 

Successive  Approxiraatioris 

Vi'e  may  use  Picard's  method  of  successive  approximations  to 
establish  the  existence  of  a  solution  of  the  system  of  equations 
(3*^)  and  to  provide  a  practical  computational  scheme.  As  our  initial 
approximation,  ^  decision  to  transform  the  system 
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directly  from  state  i  to  state  N.  Of  course,  if  no  such  direct 
link  exists  ve  etssune  that  the  time  that  elapses  is  M,  a  suitably 
large  number, 

(l)  Uj^^  ^  =  ^iN^  ^  ~  1,2, ...,N. 


The  higher  order  approximations  are  obtained  In  the  usual  way. 


(2)  { 


I  .  min  I  .  -  1, 


Jjfl 

=  0, 


for  k  =  0,1,2,....  The  physics  of  the  problem  enables  us  to  see 
some  of  the  propeirtles  of  the  successive  approximations.  For  example, 
since 


(3) 


min  {t .  . 


1,2, ... ,N  *  1, 


0, 


ve  see  that 

(4)  =  the  minimal  time  to  transform  the  system  frcm  state 

i  to  state  N  via  at  most  one  intenaedlate  state, 
1  =  1,2,.. .,N. 


And,  in  genereil, 

(k) 

(5)  ^i'  ^  minimal  time  to  transform  the  system  from  state 

i  to  state  N  via  at  most  k  Intermediate  states. 
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From  this  it  follows  that  the  approximations  are  monotone  decreasing, 

(6)  ss  1  =  1,2, 

which  Is  easy  to  establish  via  Induction.  Since  ^  Is  bounded 
from  below, 

(7)  a  0,  1  =  1,2,..., N, 

the  convergence  of  the  approximating  sequence  is  established.  As  a 
matter  of  fact,  though,  an  optlmsil  trajectory  from  any  state  1  to 
state  N  has  at  most  N  -  2  Intermediate  states,  since  an  optimal 
trsOectory  does  not  cross  Itself  to  form  a  loop,  mius  the  conver¬ 
gence  of  the  process  Is  assumed  sifter  at  most  N  -  2  stages.  !n>at 
the  limiting  values  satisfy  eqixations  (3.^)  Is  clear. 

6.  Observations  on  the  Approximation  Scheme 

Itoe  of  the  equations  (^.2)  involves  only  addition  and  conqparlson 
of  nxonbers,  two  operations  for  which  a  digltsd  computer  is  well- 
suited.  Furthermore,  in  calculating  the  vsdue  of  only  the 

i-th  row  of  the  matrix  vector  (u^^^^,U2^^^,...,Ujj^^^) 

need  be  In  high-speed  storage.  In  this  way  jiroblems  for  which  N 
Is  of  the  order  of  several  thousand  can  be  solved  by  an  IBl-7090  in 
several  minutes.  Efficient  programs  will  exploit  specisO.  features 
of  a  given  problem.  This  is  certainly  true  if  each  state  is  directly 
connected  to  only  several  of  Its  neighbors.  In  some  instances  the 
values  of  the  successive  approximations  are  more  Important  than  the 
solution  of  the  original  problem.  If  the  optimal  trajectory  from 
state  1  to  state  N  has  many  Intermediate  atates,  then  It  might 
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require  a  co2iQ>lex  instrumentation  to  achieve,  so  that  a  knowledge  of 
both  the  successive  approoclmations  and  the  limiting  values  may  be  of 
i]iQ>ortance  in  designing  a  control  system. 

Finelly,  let  us  note  that  our  solution  consists  not  so  much  in 
the  production  of  the  values  u^,U2, . . . ,Ujj,  as  in  the  knowledge  of 
the  value  of  j  which  minimizes  the  exjxression  t .  .  +  u.,  for  each 

i  J  J 

value  of  i.  This  is  precisely  the  knowledge  that  is  required  for 
determining  optimal  feedback  control. 


7.  Other  Approaches 

Many  other  approaches  to  this  problem  have  been  devised,  in 
particvileir  by  Dantzlg  [12]  and  Ford  and  Fulkerson  [13].  In  peurticu- 
lar  it  is  possible  to  fan  out  from  the  destination  and  determine, 
one  after  the  other,  the  nearest,  second  nearest,  and  so  on,  states 
to  the  terminal  state. 

A  variety  of  analogue  devices  can  be  used.  Some  of  these  are 
described  in  the  paper  [l4],  where  many  references  cure  provided. 


8.  Arbitrary  Terminal  States 

In  the  event  that  we  wish  to  determine  optimal  trajectories  from 
any  inlticLl  state  to  any  terminal  state,  we  might  apply  the  procedures 
referred  to  above  N  times,  in  each  case  letting  a  different  state 
be  the  desired  terminal  state.  Alternatively,  we  may  let 


(1) 


s  the  time  to  transform  a  system  in  state  1  into 
state  J  using  a  trajectory  with  at  most  k 


intermediate  states 
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Using  the  principle  of  optimality  ve  see  that 
(2)  -  Mn  (uj*)  .  1  *  J. 

Since  k  will  be  at  most  N  -  1,  and  since  ve  can  easily  determine 
the  matrices  )» •  •  •  * 

the  problem  is  readily  handled,  at  least  ccorputationally. 

9.  preferred  Suboptlmal  Thrajectories  [15] 

One  of  the  key  difficulties  in  applying  these  ideas  to  a  con¬ 
crete  physical  sitiiatlon  lies  in  deciding  on  the  number  and  nature 
of  the  physical  states  of  the  system  to  be  considered.  In  particular 
if  the  number  of  states  chosen  is  too  small,  l.e.,  the  grid  in  phase 
space  is  too  coarse,  the  time  to  traverse  a  second  shortest  path 
may  be  considerably  larger  than  the  time  to  traverse  an  optimal 
path.  On  the  other  hand,  if  these  times  eu:*e  not  too  different,  one's 
confidence  in  the  reasonableness  of  the  mathematlceil  model  of  the 
physlced.  process  may  be  Increeused. 

In  addition,  ve  must  recognize  that  even  if  we  determine  an 
exact  solution  to  the  mathematical  solution,  due  to  the  neglect  of 
a  variety  of  physical  factors  we  have  only  an  approximate  solution 
of  the  physical  solution.  If  a  mathematically  optimal  trajectory 
which  we  have  found  is  unsatisfactory  from  the  physical  vievpoint, 
we  may  either  find  a  near  optimal  trajectory  for  the  mathematical 
problem,  in  hopes  that  it  will  be  better  from  the  physical  vievpoint, 
or  ve  may  reformulate  the  mathematical  problem.  Let  us  show  how  ve 
may  determine  second  best  trajectories,  once  having  detexmlned 
optimal  trajectories  to  the  tezminal  state  N. 


IP 


Under  the  asGuraption  that  there  is  at  least  one  trajectory  from 
state  i  to  state  N  that  is  not  optimal,  i  =  1,2,...,N  -  1,  let 
us  define  the  variables  v^,  i  =  1,2, ...,N  -  1, 

(1)  takes  to  transform  a  system  from 
state  i  to  state  N  using  a  second  best  trajectory 
i  =  1,2,...,N  -  1, 

0. 

Of  course,  we  are  assuming  that  there  are  no  loops  in  the  trajectory 
Next  notice  that  If  we  make  the  decision  to  transform  the  system 
from  state  1  directly  to  state  J,  then  the  continuation  must 
be  along  a  trajectory  from  state  J  to  state  N  which  is  either 
optimal  or  second  best,  fnils  leads  to  the  relations 

(2)  *  mlng  [t^j  +  +  Vj],  1  •  1,2,..., N  -  1, 

Vn=  0, 

where  we  have  vised  the  notation 

(3)  min2(a^,a2,  ...,aj^)  =  the  second  smallest  of  aj^,a2, . .  .,ajj 

(under  the  assumption  that  they  are 
not  all  equal). 

Generalizations  are  given  in  the  paper  [15],  and  are  discussed  at 
length  in  [l6]. 

Pollack  [17]  has  also  observed  that  we  may  find  second  best 
paths  ty  first  finding  the  optimal  trajectory  (assumed  unique)  from 
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state  i  to  state  N.  Then  we  eliminate  the  first  link  in  the 
optimal  path  from  the  network  and  determine  ein  optimal  trajectory 
from  state  i  to  state  N  using  only  the  remaining  links  in  the 
network.  Then  this  is  done  for  each  of  the  remaining  links  in  the 
optl.mal  trajectory  being  considered.  Since  there  are  at  most  N  -  1 
such  links  in  the  optimal  trajectory,  at  most  N  -  1  such  problems 
need  be  solved.  A  trajectory  which  yields  the  smallest  of  the 
numbers  so  found  is  a  second  shortest  trajectory.  The  proof  is 
given  by  noting  that  a  second  shortest  trajectory  must  differ  from 
an  optimal  trajectory  in  at  least  one  link.  With  this  method  there 
Is  no  difficulty  concerning  the  possible  formation  of  trajectories 
with  loops. 

10.  A  Stochastic  Time-optimal  Control  Process  [l4] 

Next  let  us  ctssume  that  the  physical  situation  Is  such  that 

the  time  Involved  In  transforming  the  system  from  state  1  to  state 

j,  t^j.  Is  not  known  precisely.  Suppose,  though,  that  we  may 

consider  it  to  be  a  random  variable  with  a  known  probability  density 

function  Pj,(t).  This  Is  a  great  assumption  which  may  or  may  not 
^  J 

be  justified  in  a  given  situation.  Furthermore,  we  shall  assume 
that  the  time  to  traverse  any  link  in  a  trajectory  is  independent 
of  the  time  to  traverse  any  other  link  in  the  trajectory,  another 
severe  restriction. 

Under  the  assuc^jtions  just  stated,  our  aim  is  to  find  the 
optimal  feedback  control  decision  to  make  when  the  system  is  in 
state  i.  Let  us  now  explain  carefully  what  we  mean  by  this.  We 
shall  assume  that  our  objective  is  to  maximize  the  probability  of 


transfomlng  the  system  into  the  desired  state  N  in  a  time  t  or 
less,  \diere 

(1)  t  4  0. 

^e  sequence  of  operations  to  be  carried  out  Is  this :  First  the 
current  state  of  the  system  and  the  time  are  measured  (by  sensing 
equipment).  Secondly,  a  decision  Is  made  as  to  the  next  state  that 
the  system  Is  to  occupy.  Then  a  rcuidom  time  elapses  until  the 
system  reaches  this  state.  N^t  a  meeisxirement  Is  made  of  the  new 
time  and  state.  On  the  basis  of  this  knowledge  of  the  new  state 
and  the  new  time,  a  decision  Is  made  as  to  what  the  next  state  Is  to 
be,  and  so  on.  Notice  peurtlcularly  that  we  do  not  attempt  to  lay 
out  the  entire  sequence  of  decisions  leading  from  state  1  to  state 
N  Initially  (as  would  be  the  case  for  open  loop  control);  rather 
we  observe,  decide,  and  act  over  and  over  again.  We  aim  to  determine 
the  optimal  decision  to  make  under  given  circumstances,  l.e.,  for  a 
given  state  and  given  time  remaining  In  the  process.  While  for 
detemlnlstlc  processes  open  loop  and  closed  loop  (feedback)  control 
lead  to  Identical  results,  for  stochastic  decision  processes  they 
are  conceptually  quite  different. 

Let  us  now  define  the  functla:is  u^(t),  1  >  1,2, ...,N,  by 

the  equations 

(2)  Uj^(t)  =  the  probability  of  transforming  a  system  from  the 

Initial  state  1  to  the  desired  terminal  state  N 

In  time  t  or  less  using  an  optimal  feedback 

control  policy. 
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If  we  use  the  principle  of  optimality  once  again,  ve  can  vrlte  the 

equations 

/  t 


(3) 


u^(t)  = 


max 


p^j(t-s)Uj(s)dB, 


i  ^  *  lif 


Bqviatlons  (3)  appear  to  be  quite  difficult  to  handle  both 
analytically  and  caitq>utation€LLly.  Ube  appeeirance  of  the  convolution 
Integrals  suggests  use  of  Laplace  transforas,  but  the  occurrence  of 
the  maximum  operator  militates  against  this.  The  maximum  transfom 
discussed  by  Bellman  and  Karush  [18]  and  others  might  be  xiseful. 

Some  other  closely  related  stochastic  control  processes  eure 
discussed  In  references  [l4]  and  [19] • 


11.  MLnlmax  Control  Processes 

In  some  circumstances,  such  as  ve  have  discussed  In  Sec.  1,  it 
Is  desirable  to  transform  a  system  from  an  Initial  state  to  a 
terminal  state  In  such  a  vay  as  to  minimize  the  maximum  stress  to 
which  the  system  Is  exx>osed  durliig  the  course  of  the  process.  We 
shall  refer  to  such  control  processes  as  "mlnimax  control  processes." 
Once  again  ve  shall  consider  a  system  which  may  be  In  any  of  a  finite 
number  of  states,  N,  the  states  being  nvuabered  frcm  1  to  N. 

Ve  consider  state  N  to  be  the  desired  terminal  state.  When  the 
system  Is  transformed  directly  from  state  1  to  state  j,  a 
maxiimim  stress,  Sj  Is  encountered.  Thus  the  stress  s  Is 
associated  with  the  link  (l^j).  Our  basic  problem  consists  In 
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finding  a  treijectozy  from  state  i  to  state  N  vhlch  is  such  that 
the  maximum  stress  along  this  trajectory  is  as  small  as  possible. 

12.  Use  of  Functional  Equations 

Let  us  Introduce  the  variables  u^^  1  »  1,2, ...,N  -  1,  by  the 

relations 

(1)  ^1  maximum  stress  edong  an  optimal  trajectory  from 

state  1  to  the  teimilnal  state  N,  1  >  1,2, ...,N  -  1, 

“m- 

Then  \ise  of  the  principle  of  optimality  immediately  leads  to  the 
relations 

(2)  ==  min  {max(8^j,Uj)),  i  »  l,2,.ee,N  -  1, 

"k-  ®- 

These  are,  of  course,  the  analogues  of  eqxiatlons  (3>^).  Ibe  results 
of  the  following  sections  could  be  obtained  directly  from  these 
equations.  Rather  than  pursue  this  path,  ve  shall  keep  the  netwoxk 
itself  In  the  foreground. 

13.  A  Specled  Case 

A  very  liig>ortant  and  Interesting  speclcd  case  arises  when  ve 
stipulate  the  reversibility  equality 

8  a  8  • 

ij  Ji 


(1) 
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Thus  one  stress^  s  ,  is  connected  with  the  arcs  (i,j)  and 
^  J 

We  can  carry  through  the  analysis  in  some  detail  and  establish 
a  relationship  with  a  seemingly  unrelated  problem  Ln  graph  theory. 

We  first  observe  that  the  maximum  stress  encountered  along 
any  trajectory  is  one  of  the  numbers  us  then  arrange 

the  (positive)  numbers  in  ascending  order  of  magnitude  8uad 

denote  this  sequence  by  s^,S2, ...,3^  (R  s  (n/2)(N  +1)).  Call  the 
corresponding  arcs  For  convenience,  we  assume  that 

the  stresses  associated  with  the  eurcs  sure  all  different  from  one 
Euiother,  a  condition  which  can  be  attained  by  adding  suitably  small 
quantities  to  the  stresses  given,  if  necessary.  Then  we  observe 
that  the  states  joined  by  the  arc  having  the  stress  s^  cannot  be 
Joined  by  any  trajectory  with  a  smaller  maximum  stress,  mils  arc, 

Sj^,  constitutes  an  optimal  trajectory  for  those  states.  Nfext  we 
observe  that  the  states  Joined  by  the  arc  of  stress  Sg  are  also 
Joined  optimally  by  this  arc,  Sg. 

The  situation  becomes  a  little  more  complicated  insofar  ets  the 
states  Joined  by  arc  are  concerned.  If  the  arcs  S^,  Sg  and 

do  not  form  a  loop,  then  the  states  joined  by  arc  are  joined 

optimally  by  it.  However,  if  eurcs  Sg,  and  S^  do  form  a  loop, 

then  the  arcs  S^  and  Sg,  and  not  S^,  connect  these  states 
optimally. 

Furthermore,  we  see  that  if  we  continue  this  process  of  se¬ 
lecting  arcs  from  the  sequence  of  arcs  Si,Sg,...,Sp,  making  sure 
that  no  arc  selected  forms  a  loop  with  any  of  the  arcs  already 
selected,  we  shall  eventually  select  N  -  1  arcs  containing  no  loops. 
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We  shall  thus  have  foraed  a  particular  spanning  tree  in  the  network, 
suid  the  unique  trajectory  in  this  tree  which  connects  any  two  states 
is  the  minimax  trajectory  between  those  states. 

14.  Minimal  Spanning  Trees 

But  the  construction  which  we  have  Just  indicated  is  well  known 
to  provide  the  solution  to  smother  seemingly  unrelated  problem: 

Under  the  conditions  stated  in  Sec.  13,  find  the  tree  for  which  the 
sum  of  the  stresses  in  its  branches  (sires)  is  as  small  as  possible. 
This  tree  is  called  the  minimal  spsmning  tree.  In  1956  Kruskal  [20] 
showed  that  the  construction  given  solves  this  problem.  Vsurious 
other  algorithms  are  given  in  references  [20,l4]. 

The  algorithm  described  is  not  satisfactory  from  the  computa¬ 
tional  viewpoint,  for  testing  to  see  whether  or  not  an  arc  completes 
a  loop  when  added  to  another  set  of  arcs  can  be  quite  time-consuming 
Prim  [21]  has  given  some  very  effective  computational  procedures. 

15.  Comments  and  Interconnections 

Our  result  on  minimax  control  processes,  subject  to  the  restric 
tions  in  Sec.  13,  may  be  formulated  thusly:  optimal  minimeoc  tra¬ 
jectories  lie  in  the  minimal  spanning  tree.  Let  us  now  illuminate 
this  result  in  several  other  ways. 

First  we  give  another  proof.  Consider  an  optimal  minimax 
trajectory  from  state  i  to  state  N.  Suppose  that  is  is  not  the 
trajectory  from  state  i  to  state  N  which  lies  in  the  minimal 
spanning  tree.  Then  there  is  at  least  one  axe  in  this  trajectoiy 
which  does  not  lie  in  the  minimal  spanning  tree.  Denote  this  as 
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arc  A.  If  we  add  this  arc  to  the  set  of  arcs  in  the  minimal  spanning 
tree,  then  exactly  one  loop  is  formed,  say  the  loop  with  arcs 
(A,A^, But  the  following  inequality  must  hold 

(1)  fltre88(A)  i  max(8tre88(A^),8tre88(A2),...,8tre88(Aj)). 

If  it  did  not,  then  it  would  he  possible  to  lower  the  sum  of  the 
stresses  in  the  branches  of  the  minimal  spanning  tree  by  adding  arc 
A  to  the  minimal  spanning  tree  and  deleting  an  arc  for  which 

(2)  stress (A)  <  stress (A^). 

Inequality  (l)  shows  that  the  arc  (A)  in  the  supposed  m^n^ln«u^ 
trajectozy  may  be  repleused  by  exes  in  the  minimal  spanning  tree 
without  increasing  the  maximal  stress  encountered.  Since  arc  (A) 
could  be  any  arc  in  the  mlnlmax  trajectory-,  the  proof  is  complete. 

In  his  paper  [21],  prim  observed  that  the  minimal  spanning  tree 
not  only  minimizes  the  sum  of  the  stresses  in  its  branches  but  also 
minimizes,  among  all  trees,  any  monotone  Increasing  and  symmetric 
function  of  the  stresses  in  the  branches.  In  particular,  we  note. 

It  minimizes  the  function 

(3)  Sp  -  +  SgP  +  ...  + 

for  p  >  1,2,...,  and  the  limit  function 

(l^)  mBx(s^,Sg,...,8jj_j^)  -  lim  Sp, 


idilch  Is  easily  proved 
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This  suggests  that  we  consider  the  problem  of  determining  a 
trajectory  from  state  i  to  state  N  which  minimizes  the  sum  of 
the  p-th  powers  of  the  stresses  in  its  arcs.  Cleeurly  for  p  =  1  this 
is  equivalent  to  the  problem  of  determining  a  time-optimal  trajectory, 
which  was  considered  earlier.  On  the  other  hand,  in  view  of  equation 

(4) ,  for  p  sufficiently  large  this  is  the  problem  of  determining 

a  minimax  trajectory.  Mare  precisely,  we  can  show  that  for  p  suf¬ 
ficiently  large  a  trajectory  which  minimizes  the  sum  of  the  p-th 
powers  of  the  stresses  in  its  arcs  lies  in  the  minimal  spanning  tree. 
For  consider  eui  arc  S  which  is  in  the  optimal  trajectory  but  which 
does  not  lie  in  the  minimal  spanning  tree.  Let  its  stress  be  s, 
and  let  the  stresses  of  the  arcs  in  the  minimal  spanning  tree  with 
which  it  forms  a  loop  be  ... Then,  as  ve  observed  earlier, 

(5)  8  >  niax(8j^,S2,...,Sy), 
and  for  p  sufficiently  large, 

(6)  1  >  +  (Sg/sp  +  ...  +  (b^s)^, 


or 

(7) 


>  8, 


This  inequality  shows  that  the  arc  S  may  be  replaced  by  arcs  in 
the  minimal  speuming  tree  and  establishes  the  resiilt. 

In  reference  [22]  M.  PoUawk  posed  the  problem  of  determining 
a  maximum  capacity  jAth  between  two  stations  in  a  communications  net 
work  and  provided  several  solutions.  !niis  problem  is  equivalent  to 
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ours.  Furthermore^  D.  R.  Fulkerson  has  pointed  out  to  the  author 
that  T.  C.  Hu  [23]|  coomentlng  on  BoUack's  paper,  has  obteilned  our 
result  on  mlnlmax  trajectories. 

l6.  Multiple  Stresses 

It  frequently  happens  that  stresses  arise  during  a  process  from 
several  different  causes,  l.e,,  mechanical  and  thermal.  While,  In 
general,  a  mlnlmax  trajectory  for  one  Is  not  a  mlnlmax  trajectory 
for  the  other,  we  can  attempt  to  determine  trajectories  which  are 
optimal  In  the  sense  that  no  change  In  the  trajectoxy  can  lower  both 
stresses. 

On  link  (l,J)  let  the  maximal  thermal  stress  be  t .  .  and 

^  J 

the  maximal  mechanical  stress  be  m^^.  Then,  If  we  Introduce  the 
Lagrange  multipliers  and  tig,  we  can  associate  the  generalized 
stress  Sj^j, 

'ij  ■  Vij  *  Vij 

with  each  arc.  ^  letting,  e.g., 

(2)  nj^  -  1, 

ng  =  a, 

and  detezmlnlng  a  mlnlmax  trajectory  between  two  parti cixiar  states, 
for  which  the  maximum  mechanical  stress  Is,  for  exan^le,  s,  we  ceui 
guarantee,  that  among  all  trajectories  for  which  the  maximal  mechani¬ 
cal  stress  between  the  states  Is  s,  we  will  have  found  a  trajectory 
for  which  the  thezinal  stress  Is  minimal.  Then  a  parameter  study. 
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involving  the  determination  of  many  minimal  spanning  trees,  might 
yield  useful  design  information  concerning  the  trade-offs  that  are 
possible. 

17.  Discussion 

Our  primary  aim  has  been  to  show  the  close  connection  between 
several  important  classes  of  automatic  control  problems  and  graph 
theory.  These  considerations  have  raised  many  additional  questions. 
Let  us  conclude  by  stating  some  of  these. 

Formulation.  Hbw  are  we  to  decide  how  many  states  of  a  system 
need  be  considered,  what  the  stresses  or  times  are,  and  what  the 
criterion  is? 

Analytical  and  Congiutatlonal  Treatment.  Vfliat  are  the  connections 
between  the  solution  to  the  discrete  and  continuous  problems? 
other  problems,  too  varied  to  catalogue,  arise. 

Implementation .  After  the  optimal  feedback  control  decisions 
have  been  determined,  how  can  controllers  be  realized  to  carry  out 
the  programs? 

It  is  felt  that  the  answers  to  these  questions  would  provide 
both  automatic  control  and  mathematics  with  additional  Interesting 


chapters 
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